Search for: All records

Lecture videos are rapidly becoming an invaluable source of information for students across the globe. Given the large number of online courses currently available, it is important to condense the information within these videos into a compact yet representative summary that can be used for search-based applications. We propose a framework to summarize whiteboard lecture videos by finding feature representations of detected handwritten content regions to determine unique content. We investigate multi-scale histogram of gradients and embeddings from deep metric learning for feature representation. We explicitly handle occluded, growing and disappearing handwritten content. Our method is capable of producing two kinds of lecture video summaries - the unique regions themselves or so-called key content and keyframes (which contain all unique content in a video segment). We use weighted spatio-temporal conflict minimization to segment the lecture and produce keyframes from detected regions and features. We evaluate both types of summaries and find that we obtain state-of-the-art peformance in terms of number of summary keyframes while our unique content recall and precision are comparable to state-of-the-art. 

This work summarizes the results of the first Competition on Harvesting Raw Tables from Infographics (ICDAR 2019 CHART-Infographics). The complex process of automatic chart recognition is divided into multiple tasks for the purpose of this competition, including Chart Image Classification (Task 1), Text Detection and Recognition (Task 2), Text Role Classification (Task 3), Axis Analysis (Task 4), Legend Analysis (Task 5), Plot Element Detection and Classification (Task 6.a), Data Extraction (Task 6.b), and End-to-End Data Extraction (Task 7). We provided a large synthetic training set and evaluated submitted systems using newly proposed metrics on both synthetic charts and manually-annotated real charts taken from scientific literature. A total of 8 groups registered for the competition out of which 5 submitted results for tasks 1-5. The results show that some tasks can be performed highly accurately on synthetic data, but all systems did not perform as well on real world charts. The data, annotation tools, and evaluation scripts have been publicly released for academic use. 

We present a study on automated analysis of phase diagrams to aid the materials science community that attempts to lay the groundwork for a large-scale, searchable, digitized database of phases of a wide variety of materials at different physical conditions and compositions. For this work, we concentrate on around 80 thermodynamic phase diagrams of binary metallic alloy systems which give phase information of alloys at varied temperatures and mixture ratios. We use image processing techniques to isolate phase boundaries and subsequently extract areas of the same phase. Simultaneously, document analysis techniques are employed to recognize and group the text used to label the phases; text present along the axes is identified so as to map image coordinates (x, y) to physical coordinates. Labels of unlabeled phases are inferred using standard rules. Once a phase diagram is thus digitized we are able to provide the phase of all materials present in our database at any given temperature and alloy mixture ratio. Using the digitized data, more complex queries may also be supported in the future. We evaluate our system by measuring the correctness of labeling of phase regions.